Papers with modular framework
FreeEval: A Modular Framework for Trustworthy and Efficient Evaluation of Large Language Models (2024.emnlp-demo)
Copied to clipboard
Zhuohao Yu, Chang Gao, Wenjin Yao, Yidong Wang, Zhengran Zeng, Wei Ye, Jindong Wang, Yue Zhang, Shikun Zhang
| Challenge: | Large language models (LLMs) have revolutionized natural language processing with impressive performance across various tasks. |
| Approach: | They propose a framework for automated evaluations of large language models . they open-source their code at https://github.com/WisdomShell/FreeEval . |
| Outcome: | The framework is open-source and can be used to develop and validate new evaluation methods. |
DeepA2: A Modular Framework for Deep Argument Analysis with Pretrained Neural Text2Text Language Models (2022.starsem-1)
Copied to clipboard
| Challenge: | ArgumentAnalyst is a multi-dimensional, modular framework for performing deep argument analysis using existing pre-trained language models (PTLMs). |
| Approach: | They propose a multi-dimensional, modular framework for performing deep argument analysis using current pre-trained language models (PTLMs) ArgumentAnalyst reconstructs argumentative texts as valid arguments by inserting missing premises and conclusions, formalizing inferences, and coherently linking the reconstruction to the source text. |
| Outcome: | The proposed framework can replicate established heuristics, cope with plurality of correct solutions, and exploit higher-order evidence. |
lingvis.io - A Linguistic Visual Analytics Framework (P19-3)
Copied to clipboard
Mennatallah El-Assady, Wolfgang Jentner, Fabian Sperrle, Rita Sevastjanova, Annette Hautli-Janisz, Miriam Butt, Daniel Keim
| Challenge: | Using a modular framework, linguistic visual analytics applications can be rapidly prototypized using a web-based framework. |
| Approach: | They propose a modular framework for rapid prototyping of linguistic, web-based, visual analytics applications. |
| Outcome: | The proposed framework supports rapid prototyping of linguistic, web-based, visual analytics applications. |
FS-DAG: Few Shot Domain Adapting Graph Networks for Visually Rich Document Understanding (2025.coling-industry)
Copied to clipboard
| Challenge: | Recent advances in vision-language models have significantly enhanced performance across various natural language processing and computer vision tasks. |
| Approach: | They propose a few shot domain adapting graph (FS-DAG) that leverages domain-specific and language/vision specific backbones within a modular framework to adapt to diverse document types with minimal data. |
| Outcome: | The proposed model is highly performant with less than 90M parameters, making it well-suited for complex real-world applications for information extraction tasks where computational resources are limited. |
SentSpace: Large-Scale Benchmarking and Evaluation of Text using Cognitively Motivated Lexical, Syntactic, and Semantic Features (2022.naacl-demo)
Copied to clipboard
| Challenge: | SentSpace provides a framework for streamlined evaluation of textual input. |
| Approach: | They describe the design of SentSpace and demonstrate an example use case . they use a web interface for interactive visualization and comparison with large corpora . |
| Outcome: | The framework provides a common framework for evaluation and visualization. |
ReasonGraph: Visualization of Reasoning Methods and Extended Inference Paths (2025.acl-demo)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) reasoning processes are complex and lack of organized visualization tools creates barriers to understanding, evaluation, and improvement. |
| Approach: | They propose a web-based platform for visualizing and analyzing LLM reasoning processes. |
| Outcome: | The proposed platform shows high parsing reliability, efficient processing, and excellent usability across various downstream applications. |
AgentDiagnose: An Open Toolkit for Diagnosing LLM Agent Trajectories (2025.emnlp-demos)
Copied to clipboard
| Challenge: | Large Language Model (LLM) agents produce rich, multi-step trajectories that interleave observations, internal reasoning, and tool actions. |
| Approach: | They propose an open-source framework for diagnosing agent trajectories that quantifies five core agentic competencies and a visualization module that highlights trajectory semantics. |
| Outcome: | The proposed framework is extensible and compatible with most agent trajectories. |
Translate First Reorder Later: Leveraging Monotonicity in Semantic Parsing (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing approaches that model alignments between sentences fail at compositional generalization tasks, resulting in a resurgence of such approaches. |
| Approach: | They propose a two-step approach that first translates input sentences monotonically and then reorders them to obtain the correct output. |
| Outcome: | The proposed approach improves compositional generalization over existing models and other approaches that exploit gold alignment annotations. |
Efficient Out-of-Scope Detection in Dialogue Systems via Uncertainty-Driven LLM Routing (2025.acl-industry)
Copied to clipboard
| Challenge: | Out-of-scope (OOS) intent detection is critical in task-oriented dialogue systems . without effective OOS detection, such inputs could lead to incorrect responses, reduced user trust, and eventual system failures. |
| Approach: | They propose a modular framework that combines uncertainty modeling with fine-tuned large language models (LLMs) their method yields state-of-the-art results on key OOS detection benchmarks . |
| Outcome: | The proposed framework yields state-of-the-art results on key OOS detection benchmarks including real-world OOS data. |
A Study of the Importance of External Knowledge in the Named Entity Recognition Task (P18-2)
Copied to clipboard
| Challenge: | Existing studies have shown that external knowledge is important for Named Entity Recognition . |
| Approach: | They propose a modular framework that divides knowledge into four categories according to depth . they show the effects when incrementally adding deeper knowledge . |
| Outcome: | The proposed framework outperforms agnostic frameworks with more external knowledge . the proposed frameworks outperformed agrarian frameworks on two standard datasets . |
ReTraCk: A Flexible and Efficient Framework for Knowledge Base Question Answering (2021.acl-demo)
Copied to clipboard
| Challenge: | Existing neural semantic parsing methods for knowledge base question answering are lacking . a generic and extensible framework is lacking for KBQA. |
| Approach: | They propose a neural semantic parsing framework for large scale knowledge base question answering . they propose 'retriever-transducer-checker' framework that provides a retriever and a transducer . |
| Outcome: | The proposed framework is ranked at top1 overall performance on the GrailQA leaderboard and achieves competitive performance on typical WebQuestionsSP benchmark. |
Towards Enforcing Company Policy Adherence in Agentic Workflows (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) agents are transforming business processes with minimal human oversight. |
| Approach: | They propose a deterministic, transparent, and modular framework for enforcing business policy adherence in agentic workflows. |
| Outcome: | The proposed framework shows encouraging preliminary results in policy enforcement on the -bench Airlines domain. |
OmniEvent: A Comprehensive, Fair, and Easy-to-Use Toolkit for Event Understanding (2023.emnlp-demo)
Copied to clipboard
| Challenge: | Event understanding is fundamental for humans to understand the world. |
| Approach: | They propose an event understanding toolkit called OmniEvent that is comprehensive and fair . it supports mainstream modeling paradigms and the processing of 15 widely-used datasets . |
| Outcome: | The toolkit supports mainstream modeling paradigms and the processing of 15 widely-used English and Chinese datasets. |
GenKnowSub: Improving Modularity and Reusability of LLMs through General Knowledge Subtraction (2025.acl-short)
Copied to clipboard
| Challenge: | Large language models (LLMs) struggle with zero-shot generalization due to entanglement of general knowledge and task-specific adaptations. |
| Approach: | They propose a modular framework that disentangles general knowledge and adaptations by constructing a library of task-specific LoRA modules alongside a general-domain LoRA. |
| Outcome: | The proposed framework disentangles general knowledge and task-specific adaptations . it generates residual modules that focus more exclusively on task-relevant information . |
Deconstructing Instruction-Following: A New Benchmark for Granular Evaluation of Large Language Model Instruction Compliance Abilities (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for ensuring Large Language Models (LLMs) follow complex instructions fail to reflect real-world use or isolate compliance from task success. |
| Approach: | They propose a modular framework that uses a dynamically generated dataset with up to 20 application-oriented generation constraints to enable a granular and independent analysis of LLM instruction compliance. |
| Outcome: | The proposed framework reveals that compliance is not a monolithic capability but varies significantly with constraint type, quantity, and position. |
DANLI: Deliberative Agent for Following Natural Language Instructions (2022.emnlp-main)
Copied to clipboard
Yichi Zhang, Jianing Yang, Jiayi Pan, Shane Storks, Nikhil Devraj, Ziqiao Ma, Keunwoo Yu, Yuwei Bao, Joyce Chai
| Challenge: | Recent work on embodied AI agents that can perform tasks by following human language instructions is limited by reactive methods, which are insufficient for long-horizon complex tasks. |
| Approach: | They propose a neuro-symbolic deliberative agent that, while following language instructions, proactively applies reasoning and planning based on its neural and symbolic representations acquired from past experience. |
| Outcome: | The proposed agent achieves greater than 70% improvement over reactive baselines on the challenging TEACh benchmark. |
MAViS: A Multi-Agent Framework for Long-Sequence Video Storytelling (2026.eacl-long)
Copied to clipboard
| Challenge: | Experimental results show that MAViS achieves state-of-the-art performance in assistive capability, visual quality, and video expressiveness. |
| Approach: | They propose an end-to-end multi-agent collaborative framework for long-sequence video storytelling that orchestrates specialized agents across multiple stages. |
| Outcome: | The proposed framework achieves state-of-the-art performance in assistive capability, visual quality, and video expressiveness. |
DeFine: Decision-Making with Analogical Reasoning over Factor Profiles (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models are ideal for decision-making, but they can be difficult to process when they are verbose and include repetition, hedging, and vagueness. |
| Approach: | They propose a framework that constructs probabilistic factor profiles from complex scenarios and integrates them with analogical reasoning to guide LLMs in making decisions in new situations. |
| Outcome: | The proposed framework separates the tasks of quantifying uncertainty and incorporating it into LLM decision-making. |
Modular Pluralism: Pluralistic Alignment via Multi-LLM Collaboration (2024.emnlp-main)
Copied to clipboard
Shangbin Feng, Taylor Sorensen, Yuhan Liu, Jillian Fisher, Chan Young Park, Yejin Choi, Yulia Tsvetkov
| Challenge: | Existing alignment paradigms for large language models learn an averaged human preference and struggle to model diverse preferences across cultures, demographics, and communities. |
| Approach: | They propose a modular framework that "plugs" into a base LLM a pool of smaller but specialized community LMs where models collaborate in distinct modes to support three modes of pluralism: Overton, steerable, and distributional. |
| Outcome: | The proposed framework “plugs into” a base LLM a pool of smaller but specialized community LMs, where models collaborate in distinct modes to support three modes of pluralism: Overton, steerable, and distributional. |
Reducing Hallucinations in Language Model-based SPARQL Query Generation Using Post-Generation Memory Retrieval (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are susceptible to hallucinations and out-of-distribution errors when generating KG elements, such as Uniform Resource Identifiers (URIs). |
| Approach: | They propose a SPARQL query-generating framework that uses natural language placeholders and a non-parametric memory module to retrieve and resolve the correct KG URIs. |
| Outcome: | The proposed framework significantly enhances query correctness across various LLMs, datasets, and distribution shifts while achieving the near-complete suppression of URI hallucinations. |
CUE Vectors: Modular Training of Language Models Conditioned on Diverse Contextual Signals (2022.findings-acl)
Copied to clipboard
| Challenge: | Using contextual universal embeddings, we train neural language models on one type of contextual data and adapts to novel context types. |
| Approach: | They propose a framework to modularize the training of neural language models that use diverse forms of context by eliminating the need to jointly train context and within-sentence encoders. |
| Outcome: | The proposed framework trains LMs on one type of contextual data and adapts to novel context types. |
Infogent: An Agent-Based Framework for Web Information Aggregation (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing web navigation tasks evaluate web agents on task completion basis . however, information aggregation tasks have received relatively little attention . |
| Approach: | They propose a web navigation framework that uses three components for web information aggregation. |
| Outcome: | The proposed framework beats existing SOTA search framework by 7% under Direct API-Driven Access on FRAMES and improves over an existing information-seeking web agent by 4.3% under Interactive Visual Access on AssistantBench. |
Rule-based Morphological Inflection Improves Neural Terminology Translation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Current approaches to incorporating terminology constraints in machine translation (MT) typically assume that the constraint terms are provided in their correct morphological forms. |
| Approach: | They propose a framework for incorporating lemma constraints in machine translation . they use a cross-lingual inflection module that inflects the target lemmo constraints based on the source context. |
| Outcome: | The proposed framework outperforms existing methods with lower training costs and linguistic knowledge in domain adaptation and low-resource MT settings. |
Train Once, Use Flexibly: A Modular Framework for Multi-Aspect Neural News Recommendation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent neural news recommenders (NNRs) extend content-based recommendation by aligning additional aspects between candidate news and user history or diversifying recommendations w.r.t. these aspects require retraining of the model with a modified objective. |
| Approach: | They introduce a modular framework for multi-aspect neural news recommendation that supports on-the-fly customization over individual aspects at inference time. |
| Outcome: | The proposed framework outperforms state-of-the-art NNRs on both content-based recommendation and single- and multi-aspect customization. |
Breaking Down and Building Up: Mixture of Skill-Based Vision-and-Language Navigation Agents (2026.acl-long)
Copied to clipboard
| Challenge: | Vision-and-Language Navigation (VLN) is a subfield of embodied AI that integrates natural language understanding, visual perception, and sequential decision-making to allow autonomous agents to navigate and interact within visual environments. |
| Approach: | They propose a modular framework that introduces structured, skill-based reasoning into Transformer-based VLN agents. |
| Outcome: | The proposed framework decomposes navigation into atomic skills handled by a specialized agent. |
TaPERA: Enhancing Faithfulness and Interpretability in Long-Form Table QA by Content Planning and Execution-based Reasoning (2024.acl-long)
Copied to clipboard
| Challenge: | Long-form table question answering often generates paragraph long and complex answers . a prevalent and concerning issue is hallucination, where models generate answers that are coherent yet factually incorrect or irrelevant to the input context. |
| Approach: | They propose a modular framework that decomposes the whole process into three sub-modules . framework produces a QA-based plan first, followed by generating an answer conditioned on this plan . human evaluation results indicate the framework improves strong baselines on accuracy and truthfulness . |
| Outcome: | The proposed framework improves accuracy and truthfulness on the FeTaQA and QTSumm datasets. |
SLiNT: Structure-aware Language Model with Injection and Contrastive Training for Knowledge Graph Completion (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) lack structural information and semantic context to infer missing entities . large language models often lack structural signals to infuse missing entities into knowledge graphs . |
| Approach: | a modular framework integrates structural information and semantic context into a frozen LLM backbone for link prediction. |
| Outcome: | a new framework integrates KG-derived structural information and semantic context to infer missing entities. |
Small LLMs Are Weak Tool Learners: A Multi-LLM Agent (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have revolutionized natural language processing with impressive capabilities, but they lack domain specificity, real-time information and face challenges in solving specialized problems. |
| Approach: | They propose a multi-LLM approach that decomposes the aforementioned capabilities into a planner, caller, and summarizer. |
| Outcome: | The proposed model outperforms existing models by demonstrating its effectiveness and advantages in tool learning. |
NEXUS: Network Exploration for eXploiting Unsafe Sequences in Multi-Turn LLM Jailbreaks (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to large language models suffer from limited exploration of adversarial space . multi-turn jailbreaks that distribute malicious intent across benign exchanges are vulnerable . NEXUS aims to exploit the adversarials of LLMs for maximum effectiveness in jailbreak scenarios . |
| Approach: | They propose a framework for constructing, refining, and executing optimized multi-turn attacks . NEXUS builds a semantic network of thought that captures a comprehensive representation of the adversarial space . |
| Outcome: | NEXUS can achieve higher attack success rate than state-of-the-art approaches . it builds a semantic network of thought that captures a comprehensive representation of the adversarial space . |
XLSR-MamBo: Scaling the Hybrid Mamba-Attention Backbone for Audio Deepfake Detection (2026.findings-acl)
Copied to clipboard
| Challenge: | Advanced speech synthesis technologies have enabled highly realistic speech generation, posing security risks that motivate research into audio deepfake detection (ADD). |
| Approach: | They propose a modular framework that integrates an XLSR front-end with synergistic Mamba-Attention backbones to capture artifacts in spoofed speech signals. |
| Outcome: | The proposed framework achieves competitive performance on the ASVspoof 2021 LA, DF, and In-the-Wild benchmarks compared to other state-of-the art systems. |
One Refiner to Unlock Them All: Inference-Time Reasoning Elicitation via Reinforcement Query Refinement (2026.acl-long)
Copied to clipboard
| Challenge: | Existing alignment methods for Large Language Models (LLMs) are expensive and lack the flexibility to fully activate their latent reasoning capabilities. |
| Approach: | They propose a modular framework that treats reasoning elicitation as an inference-time alignment task. |
| Outcome: | The proposed framework outperforms baselines by 2.1% on average across diverse architectures and benchmarks. |